Towards Very Large Scale Digital Library Building in Greenstone Using Parallel Processing
نویسندگان
چکیده
As very large digital library collections become more commonplace, software tools must adapt appropriately. This paper reports on an evolution of the Greenstone Digital Library software to support parallel processing during the collection building phase. A series of experiments were conducted to first establish a basic speed-up factor, and then deconstruct the parallelisation process to understand the execution profile of the application. Several bottlenecks were identified and resolved to further improve the performance. The adaptation of Greenstone confirms that the build phase is indeed a suitable candidate for parallelisation; and suggests that parallelisation of processing is a new avenue for exploration in emerging digital library architectures.
منابع مشابه
Parallel Processing Videos in Very Large Digital Libraries
Nowhere are the ‘growing pains’ of Very Large-scale Digital Libraries more pronounced than in collections containing multimedia data. Not only do such collections contain large numbers of items, but they also push the boundaries of scale in terms of storage space and processing expense. In this paper we explore how applying parallel processing opensource libraries and techniques—previously deve...
متن کاملCoping with very large digital collections using Greenstone
The Greenstone digital library software is widely used for small to medium digital library collections, but its reputation for creating very large collections is less well established. This paper describes how Greenstone is being used to produce large newspaper collections for the National Libraries of New Zealand and Singapore, respectively. It also describes current developments that integrat...
متن کاملCustomizing Digital Library Interfaces with Greenstone
The Greenstone digital library software is intended to help users construct simple collections of information very quickly. Indeed, only a few minutes of the user’s time are needed to set up a collection based on a standard design and initiate the building process. Collections may be large—some comprise Gbytes of text; millions of documents. Furthermore, even larger volumes of information may b...
متن کاملDigital Libraries in Asian Languages - A TCL Initiative
The Greenstone Digital Library (GSDL) system, developed by the New Zealand Digital Library (NZDL) Consortium at the University of Waikato is a suite of open-source software for building and distributing digital library collections. At the Thai Computational Linguistic (TCL) Laboratory of CRL Asia Research Center, we plan to implement and host digital libraries in several major Asian languages. ...
متن کاملEvaluating the Usability and Efficiency of the National Oil Corporation - Digital Library in Libya
This paper presents a preliminary discussion on some of the results from a survey aimed to explore, describe and explain some of the usability characteristics in digital library evaluation in the Libyan context. The study is framed in the evaluation of a bilingual digital library: The National Oil Corporation (NOC) Digital Library in Libya. It is worth mentioning in this context that this study...
متن کامل